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ABSTRACT 

A review of current research on workplace literacy 
programs reveals few programs reporting rigorous evaluations. 
Assessments are often limited to the completion of questionnaires, 
surveys of program participants , and anecdotal reports of 
effectiveness. Occasionally, a standardized reading test provides an 
indication of learner gains. Only a few evaluations provide follow-up 
data on the impact of programs on job performance, retention, or 
earning power. Trends among programs for which more rigorous 
evaluations have been performed are as follows: (1) effective 
programs require significant resources in terms of learner time on 
task; (2) effective private programs report learner cost figures more 
than double those of average public programs; and (3) effective 
programs integrate basic skills training with workplace technical 
training and usually involve counseling, on-the-job training linkage, 
and analysis of the basic skills needed on learner jobs. The 
eight-volume Job Training Partnership Act evaluation plan is the most 
thorough guideline for evaluating the effectiveness of preemployment 
literacy programs. Stufflebeam's Context, Input, Process, Product 
evaluation model has been modified by outside consultants for use 
with workplace literacy programs. This model uses interviews, 
document analysis, observations, and test data to determine program 
goals, sufficiency of resources, sufficiency of learning methods, and 
evidence of goal attainment. (14 references) (YLB) 
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Abstract 



A review of current research on workplace literacy programs reveals 
few programs reporting rigorous evaluations. Assessments are often limited 
to the completion of questionnaires, surveys of program participants, and 
anecdotal reports of effectiveness. Occasionally a standardized reading test 
provides an indication of learner gains. 

Only a few evaluations provide follow-up data on the Impact of 
programs on learner Job performance, retention, or earning power. Among 
programs for which more rigorous evaluations have been performed, a few 
trends are apparent. 

1) Effective programs require significant resources in terms of 
learner time on task (i.e. 50-100 hours of instruction per average 1 
year of learner gain). 

2) Effective private programs report learner cost figures more than 
double those of average public programs (i.e $7000 vs $2800). 

3) Effective programs integrate basic skills training with workplace 
technical training. This usually involves counseling as well as 
on-the-job training linkage and analysis of the basic skills needed 
on learner Jobs. 
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Early Evidence for Effeethre Programs 

During the 1970'vS, the U.S.mllitary Functional Uteracy (FLU) Project 
collected the most extensive workplace literacy program evaluation data 
yet available. The 120 hour, six week program evaluated data for over 700 
enlisted men. Results indicated that students in the FLIT program 
demonstrated three times the improvement in Job-related reading as in 
general reading and performed Job-related reading three times better than 
comparable students in other Army and Air Force programs. This 
indicates that general re? ling training does not transfer well to Job reading 
performance and the targeted Job-literacy training is more effective than 
general training. Further, retention studies indicated that ater 8 weeks 
FLIT personnel retained 80% of their end-of-course gain in Job-related 
reading, but only 40% of their end-of-course gain in general reading 
General reading gains for the 120 hour program averaged .7 grade level 
while Job reading gains averaged 2.1 grade levels (Sticht, 1982, pp. 24-27). 

During the early 1980's, a few civilian programs attempted to Integrate 
workplace instruction with on-going technical training. Mikulecky and 
Strange (1986) report on a program to train word processor operators and 
a second program to train wastewater treatment workers. Each program 
involved extensive training time (several hundred hours) and Involved 
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some form of screening for admission. The word processor operators 
attended paid training 40 hours per week for from 14-20 weeks (until they 
were able to mnction at levels comparable to average employed word 
processor operators). The wastewater treatment training program 
involved 20 full weeks of voluntary training which alternated classroom 
training with on-the-job training and provided approximately 100 hours of 
supplementary literacy support for the least academically able of its 
workers. 

The word processor training program experienced a drop-out rate 
below 10 percent. The average learner reached job-level competence in 
an average of 20 weeks or 800 hours (the earliest trainees found 
employment in 14 weeks with a few taking nearly 28 weeks). The 
1981-82 program concluded in the middle of a major recession in which 
1/3 of cooperating companies stopped all hiring. In spite of these 
economic :V^culties, 70 percent of program participants found 
employment as word processor operators by October of 1982. Th^ 
wastewater treatment workplace literacy program focussed on the least 
literate 20 percent of workers. Nearly 1/2 passed their technical training 
post tests. The consensus of technical instructors was that less than 5 
percent would have passed without additional support. Of students 
attending special training sessions, nearly 70 percent were able to 
accurately summarize job materials in their own words by the end of 
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training. Only about 10 percent of learners demonstrated gains in general 
reading abilities and these were students wao invested 5 or more hours 
weekly outside of class on general reading materials. Retention of students 
receiving special basic skills training was higher than that of more able 
students who attended technical training only. 

Current Workplace Literacy Program Evaluations 

Research and evaluation data on the effectiveness of current workplace 
literacy programs are a bit uneven. A few programs (Halgler, 1990; 
Philippi, 1989, Hargroves, 1989, Auspuf., et al, 1989) have systematically 
collected evidence of effectiven....s. Many other program reports are 
limited to sketchy descriptions of program components, anecdotal 
recountings as indications of effectiveness, and incomplete references to 
learner performance results. 

JSEP ; Among recent workplace literacy programs which report 
e\aluation data are two programs which transfer the U. S. Army's Job Skills 
Education Program (JSEP) to civilian settings. JSEP consists of 315 
workplace basic skills lessons, the majority of which are computer based 
and which contain approximately 300 hours of instruction. The 
individualized, competency oriented, self-paced lessons are intended for 
adults functioning between fifth and eighth grade reading levels. Students 
using the JSEP system are identified by their Job specialty and their 
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instructional needs are keyed to the basic skills associated with their 
particular Jobs. 

Haigler (1990) describes a civilian modification of the military 
materials which was tested in an adult basic education setting in White 
Plains, New York and Philippi (1989) reports the evaluation results of a 
similar modification by a Mississippi business/community college 
partnership. In the White Plains program, 61 adults (31 ABE students, 20 
ESL students, and 10 GED students) spent an average of 7S.8 hours on line 
with the computer modules to complete an average of 40.5 lessons. Pre 
and post test results on the Test of Adult Basic Education fTABE) indicated 
an average gain of 1.26 grade level in reading and .94 in math. Students 
also averaged 37% gain on a 65 item test developed from learning 
materials. Gains ranged from 22% for GED students to 54% for ESL 
students (Haigler, 1990). 

Philippi (1989) reports on a JSEP adaptation in Mississippi involving 
the National Alliance of Business, Meridian Community College, and 
current employees of Peavy Electronics. The employer was interested in 
whether participation in the basic skills program would enable employees 
to better perform on the job, better prepare workers to deal with the 
introduction of new technology, and better prepare them for promotion. A 
pilot group of 63 employees, screened to be between grades 5 and 8 in 
reading ability, used JSEP lessons prescribed to match skill demands of 
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their Jobs. Literacy task analyses were used to determine Job skills 
demands. For twelve weeks, employees twice weekly attended 1 hour and 
40 minute classes. Ninety-five percent of employees missed one or fewer 
classes. The remaining students missed a maximum of three classes. 
Learners averaged 39.47 hours of Instruction with a range of 35 hours to 
40 hours. 

Though no control group was available for comparison, post-program 
interviews with supervisors indicated varjrtng degrees of program 
effectiveness. Supervisors noted Job performance improvement In 33% of 
participants. Over 20% of participants had Inquired about openings in Jobs 
requiring higher skills and/or computer operation. Supervisors indicated 
they would recommend 57% of JSEP participants for pay increases and 
60% for promotions. In terms of Impact on operations, supervisors noted 
Increased productivity and quality In the work of participants. This 
Increase was attributed to Improved ability to read gages, schematics, do 
calculations, and work in teams. Nearly half of supervisors felt their Jobs 
had become easier as a result of the JSEP program (Phlllppl, 1989, 58-59). 

One area of concern for future users of the JSEP system was hardware 
reliability. PhlUlpl notes that "the Instructor's log and the learner 
comments about JSEP management and hardware systems indicate less 
than satisfactory Interface with the system (Phlllppl, 1989, p 64). " 
Documented complaints highlighted frustrating equipment failure. 



ERIC 



5 

8 



How Effective Are Workplace literacy Programs? 

difficulties with the use of the light pen technology, as well as complexities 
and inaccuracies with computerized record-keeping system. Accessible 
technical support would seem to be key for future JSEP use. 

Federal Reserve Bank's Skills Development Center : Hargroves (1989) 
reports on a particularly extensive long-term study of the impact of a 
workplace basic skills program upon Job performanc . earnings, and 
retention. She presents the results of a 15 year comparison of Federal 
Reserve Bank basic skills trainees to a peer group of entry level workers in 
terms of: 1) effectiveness of training in helping undereducated youth catch 
up. 2) retention, 3) job performance and 4) earning power. Hargroves 
describes the Boston Federal Reserve Bank's Skills development program 
which integrates basic skills and clerical training, supervised work 
experience, and counseling. Trainees come into the program because they 
lack basic skills needed in most clerical Jobs. Though 50% of trainees have 
graduated from high school, half read at the eighth grade level or below. 
Two out of three Skills trainees attend long enough to complete an 
extensive class and on-the-job training program leading to Job placement. 

Hargroves (1989) gathered information on 207 Skills Center trainees 
from 1973 to 1988 and compared employment data to that of 301 Bank 
employees hired for entry-level positions from 1974 to 1986. Results 
indicate that several months of formal training combined with on-the-job 
experience and counseling can enable undereducated youth to catch up to 
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typical entry-level workers. Two thirds of trainees (who would not 
otherwise have been eligible for employment) were placed in Jobs. The 
trainees, on the average, stayed longer than their entry-level peers, despite 
a low unemployment rate and ample Job opportunities outside the bank in 
the late 1980's. The majority of Skills Center graduates earned as much as 
their entry-level peers who were more educated and experienced. "In 
summary, the program produced a supply of employees who were trained 
as well or better than other new entry-level employees and understood the 
Bank's employment practices; it also provided trainees to departments on 
short notice for extra clerical help (Hargroves, 1989, p. 67)." 

Several elements key to program success are highlighted. These 
include: 1) integrating basic skills, clerical skills, work experience and 
intensive counseling, 2) self-paced and often one-on-one instruction 
focussing on competence, 3) connections to community agencies for 
recruitment, and 4) good communications with Bank supervisors in order 
to develop Job placements. The program cost was Just over $7,000 per 
enrollee in 1987 which contrasts to an average of $2,800 per person in 
publicly funded adult training programs (Hargroves, 1989, p. 67). Given 
the high turnover rate in many bank positions, the more expensive private 
program with a proven track record has been Judged to be cost-effective. 

Vestibule and Pre-wnrk Basic Skills Programs! Another thorough 
recent evaluation of a pre-work literacy program is the evaluation of 
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JOBSTART In 13 diverse sites across the country (Auspos, P. and Others, 
1989). JOBSTART Is a demonstration program designed to address the 
employability problems of school dropouts. Funding comes from more 
than a dozen sources, but the program is offered primarily under the 
federal Job Training Partnership Act (JTPA). The program offers basic 
education, occupational skills training, support services, and job placement 
assistance for economically disadvantaged dropouts who read below the 
eighth-grade level. 

In the first year, 2,312 applicants were randomly assigned to either an 
experimental group receiving JOBSTART training or to a control group not 
offered JOBSTART, but receiving other available community services. 
1,401 of the original applicants were available for a 12'month follow-up 
survey. Participants averaged 132 hours of basic education. 

Comparison of JOBSTART participants to other JTPA participants 
revealed that they were more disadvantaged and received more training (6 
months vs. 3.4 months) than is typical for JTPA participants. 
Approximately 30 percent of participants reported receiving a GED within 
twelve months of entering the program (43 percent for those starting 
above a seventh grade reading level and 20 percent for those beginning 
below that level). Only 1/3 of the participants were tested for reading 
gains. The Test of Adult Basic Education was useo uO measure gains. These 
participants averaged .7 of a year gain in reading ability (average grade 6.9 
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to 7.6) after approximately 100 hours of Instruction. 

Compared to the control group, JOBSTART participants were 
substantially more likely to receive GEDs or higli school equivalency 
certificates. Job placement for JOBSTART participants was the least 
effective component of the program. In the short term, JOBSTART 
participants averaged lower earnings than control group members. The 
evaluation also discusses constraints placed upon the program by JTPA 
performance standards. Chief among these is that increasingly vendors are 
paid only if enrollees reach benchmarks in achievement including 
placement in a training-related Job. This creates an incentive for less 
training and more rapid placement. Similarly, no funds are available for 
vendors to recruit and keep records on control group participants. These 
practices have been identified as likely to discourage vendors from 
working with populations who tend to need longer and more expensive 
training and from evaluating program effectiveness in any rigorous fashion. 

Hirschoff (1988) describes another vestibule literacy project designed 
to help low-income women become blue-collar workers. Non-Traditional 
Employment for Women (NEW) was established in 1978 as more than a 
basic skills preparatory employment program for low-income women. The 
program includes Job placement services and introducing women to 
non-traditional career opportunities. In 1984, NEW began providing a 
literacy program for women with reading scores below an eighth grade 
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level. NEW applicants were steered to the literacy program if their 
reading and math scores preclude reasonable hope for acceptance Into a 
skills training or decent Jobs. 

NEW's literacy program served fifty-six students during the 1987 
fiscal year. The only evaluative process appears to be pre and post-testing 
of the students reading abilities using the Test of Adult Basic Education 
(TABE). Sixty-two point five percent of the students In the 1987 program 
"who took post-tests Improved their scores by two to three grade levels, 
while twelve point five percent gained three levels or more. All but six 
gained at least one grade level" while approximately twenty-two students 
improved enough "to enter NEW's skills training program, \welve entered 
pre-apprentlceship classes and are now earning $10 to $18 an hour in 
blue-collar positions. Eleven women took NEW's building maintenance 
course and went on to related Jobs. Two entered GED programs and 
received their certlficates"(Hlrschoff 1988, p. 10). No information was 
provided on the average learning times needed to obtain the reported 
gains. Ihe program claims it maintains no set curriculum nor fixed time 
frame, partly because applicants enter the classes at any time throughout 
the year and partly because their needs and backgrounds vary so much. 

Though data on vestibule literacy program effectiveness is limited, the 
JTPA Evaluation Design Project is In the process of developing an 
extensive guide for JTPA program evaluation. The evaluation guide is an 
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eight volume series of "evaluation tools that are useful to states and local 
service delivery areas in Judging how J1?A programs are being managed 
and how tJtiey are impacting program participants and employers" 
(Washington State Dept. of Employment Security 1986, p.l). The eight 
volumes are the following: 



Volume 1: Overview 

Volume 2: A General Planning Guide (State and local version) 

Volume 3: A Guide for Process Evaluations 

Volume 4: A Guide for Gross Impact Evaluations 

Volume 5: A Guide for Net Impact Evaluations 

Volume 6: An Implementation Manual for Net Impact Evaluations 

Volume 7: Issues related to Net Impact Evaluation 

A. Issues in Evaluation Costs and Benefits 

B. The Debate Over Experimental vs. 
Quasi-experimental Design 

Volume 8: MIS Issues in Evaluating JTPA 

Later volumes are not yet available and the evaluation guide does not 
address the constraints identified by the JOBSTART evaluation. 

Washington's Department for Employment Security (1986) reports that 
the volumes "respond to the differing needs of both state and local users ... 
designed to offer JTPA users a fairly selective yet diversified menu of 
technical assistance products to meet a variety of evaluation needs and 
interests. Taken together, these products support comprehensive 
evaluations over the JTPA planning cycle"(p.2). 

Current Program Descriptions without Extensive Evaluation Data: The 
research data bases include several descriptions of workplace literacy 
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programs developed as cooperative ventures between businesses, unions, 
communities, and educational providers. Though few thorough evaluations 
are mentioned, some Indications of program evaluation are discussed. 

Rosenfeld (1987) describes four Innovative and effective adult 
literacy programs In four employer-based sites: a university In North 
Carolina, a naval base In Tennessee, a large company In Virginia, and a 
coordinated community program in Alabama. 

Hie goal of the Physical Plant Adult Basic Education Program at North 
Carolina State University was to provide basic skills and GED education to 
university service employees. The program offers two different levels of 
basic education and a GED test preparation course. Instructors use student 
stories. ...torature, poetry, songs, cultural experiences, and history relevant 
to the Interests of the students because of the program's belief that 
workplace literacy programs do not have to teach only Job-related skills 
through Job-related materials. Gains In reading levels for the participants 
have ranged from 1.5 to 2.0 grade levels. However, no mention Is made as 
to how these gains were measured. The program lasts eleven months, and 
participants spend four hours of work-release time a week in classes. 
Rosenfeld (1987) admits that economic outcomes are difilcult to document 
and that "[ilndirect benefits to the university, according to the director, 
are more qualified employees who are less prone to be absent and to feel 
frustrated by lack of qualifications and opportunltles"(p.8). Again, no 
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evidence for these Indirect benefits is provided. There is some question of 
the transferability of reading poetry to that of reading a memo from a 
supervisor. No data related to this question were provided. Rosenfeld 
(1987) concludes that the goals of the program were "more altruistic than 
economic. It is a case more of community service than plant 
efflciency"(p.8). 

The U.S. Navy and the city of Memphis, TN developed a joint 
workplace literacy project in 1986 entitled Project Literacy . The navy had 
been conducting a short-term total immersion program called Academic 
Remedial Reading (ARR) for its recruits. No details about this program's 
effectiveness are provided. As part of the cooperative effort, the Navy's 
program was modified to fit the needs of the sixteen Memphis Sanitation 
and Parks Department employees who participated in the ten fuU 
workdays program. At the end of the ten days, it was noted that all sixteen 
learners made "increases in the reading levels ranging from 0.5 years to 
6.0 years. The average improvement was 3.6 grade levels" (Rosenfeld 1987, 
p.6). These gains were measured by comparing learner pre and 
post-program test scores on an unnamed test. Other reported js>.ins were 
improved self-esteem among the participants, a desire to continue 
improving their learning, and an "esprit de corps" which led to peer 
support and tutoring. The basis for these gains was not mentioned. 
Program reports indicate a need for testing hearing and eyesight, flexibility 
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in modifying lessons accordir.g to student needs, and having course 
materials "be relevant to adult leamers"(Rosenfeld, 1987, p5). 

The Newport News Shipbuilding and Drydock Company in Virginia 
and the local school system Joined together in 1985 to improve the 
opportunities for the company's older employees and to perform a service 
to the community. The partners received a grant from the state to design a 
Joint literacy program entitled The Reading Progkam. Curriculum 
included both work-related and personal living experience materials. 
"Participants learned about money management and health care as well as 
shipyard safety and work-related communications" (Rosenfeld, 1987, p.9). 
Based upon anecdotal reports, Virginia had given increased financial 
support to the second year of the program before a more formal evaluation 
report was completed. As of the writing of Rosenfeld's article, a formal 
evaluation was being done, but the results were not yet available. 
Therefore, no comment on evaluation practices was made. 

The Albertville City School System, four local large employers 
(Keyes Fiber, Arrow Shirt Company, Kendall Company, and the City of 
AlbertsvlUe), and the Northeast Alabama Area Adult Basic Education 
Program established the Albertville Business/Industry Educational Program 
in 1985. The program was designed for workers in area industries who 
were unlikely to be reached through regular Adult Basic Education 
programs . Each of the four participating employers worked with the 
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school system to provide their own job related materials at each of the 
worksite classrooms. Support by both plant managers and local politicians 
is reported. No information about program effectiveness or evaluation is 
presented. The only results indicated are that, "Imlore than half of the 
employees who strengthened their basic educational skills through the 
program went on to complete their GED"(Rosenfeld 1987, p. 1 1). 

The Business Council for Effective Literacy (1987) reports on the 
role of unions in providing adult basic skills education. Larnched in 
1985, the Consortium for Worker Literacy in NYC was created through the 
cooperation of eight unions (Teamsters Joint Council 16, International 
Ladies Garment Workers Union, United Auto Workers District Council 65, 
American Federation of State County and Municipal Employees District 
1707, Amalgamated Clothing and Te: " rkers Union, United Auto 
Workers Local 259, Health and Hospital Workers Union District Council 
1 199, and Hotel and Restaurant Workers Union). Consortium activities 
are reported by BCEL to compromise one-third of the total adult education 
classes provided by the New York City Board of Education. The Consortium 
also "works with CUNY (City University of New York) to develop research 
on questions of direct concern to its basic skills efforts" (Business Council 
1987, p. 5). No research results are provided, nor is there any indication of 
program practice evaluation. 

In "Adult Literacy: Industry-Based Training Programs," Fields, Hull, and 
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Sechler (1987) discuss tlie operation and evaluation of seven 
industry-based programs. As part of their research for the National Center 
for Research in Vocational Education at Ohio State University, they 
interviewed company ofQcials, plant managers, union officers, literacy 
instructors and employees to find out what happens in industiy-based 
literacy programs. 

Texas Instruments, Inc. 

The Alliance for Effective Education is a result of the coUaboraUon 
between Texas Instruments and Austin's local government and education 
facilities. Austin Community College and Texas Instruments Joined forces 
in 1983 and established the Basic Skills for Test Operator's Program which 
includes ESL (English as a Second Language) training and GED classes for 
the company's Thai and Hispanic workers, as well as other employees who 
wish to get their high school equivalency degree. Participants all must 
undergo a formal skills assessment; one test used is the Flannigan 
Industrial Series which tests for math level and related technical areas. 
Instruction is provided by the college and the teachers focus on (1) aiding 
learners in setting goals, (2) keeping courses and content flexible, (3) 
providing individualized instruction, and (4) guiding and motivating 
learners (Fields et.al. 1987, p.31). No systematic evaluation of impact on 
productivity was attempted. Some rough evaluation indicators of the 
program's success were reported by the Human Resources Department. It 
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was noted that employees who participated in the program. "(1) passed 
the test operator's program at a 100 percent success rate, while 
nonparticipants did not; (2) show improved Job satisfaction on attitude 
surveys; (3) show satisfaction with training during performance reviews; 
and (4) show improved work performance as reported by supervisors" 
(Fields et.al. 1987. p.31). How the Human Resources Department came to 
these conclusions is not indicated. Specific details on the degree of impact 
for program participants is not discussed. 

Philadelphia Hospital and Health Care District 1199C 

District 1 199C of the Nation Union of Hospital and Health Care 
was established in Philadelphia in 1974 to win for its members the right to 
advance themselves economically and socially by providing them with the 
educational opportunity needed for higher-paying Jobs in the health care 
field. Courses focus on "training at a post-high school level for 15 
occupations in the health care field" (Fields, et. al. 1987, p. 10). Training 
also concentrates on basic skills. The program serves several hospitals and 
all union members who wish to participate. Classes do not always take 
place at employees' work-sites, nor can students participate in the 
program during work hours. Commitment to the program has to be on 
workers'own time. 

Learner progress is the only aspect of the program which is 
evaluated formally. Evaluation of students' progress in basic skills classes is 
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accomplished through instructor testing and GED exam results. No 
mention Is made of the nature of these results. Fields reports some 
positive employer feedback on participant performance, but again no 
details are offered. Program effectiveness is measured by "the demand 
expressed for the fund's basic skills classes"(Fields, et. al. 1987, p. 12). In 
other words, the program is believed to be effective when it has a long 
waiting list. Some anecdotal results are reported. One such report 
Indicated "(al freak experience occurred with one Pre-Allied Health 
class"(p. 12). All of the students in this particular class showed an increase 
in their basic skills and were hired. However, none of their Jobs were in 
the health care field. Fields hypothesizes that "Christmas time tempted 
them away from training further to accepting seasonal employment"(Flelds 
et.al., p. 12). No discussion occurs of other equally viable hypotheses. For 
example: (1) the program may have been effective in helping students 
improve some basic skills but not health care related basic skills, (2) the 
health care profession did not pay enough, or (3) the only work that the 
students could get as a result of the program was seasonal work. 
Onan Corporation 

TTie Onan Corporation manufactures portable electric generators, 
and gasoline and diesel engines. The company first introduced a basic 
education class in 1982 as a result of employee frustration with the rapidly 
changing and Increasingly complicated technologies being introduced to 
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their workplace. "A needs assessment was performed by consultants, 
involving over 600 workers to determine their basic education needs and 
interests. Results prompted the plant to offer a more comprehensive 
on-site program that included basic education and pre-technical 
courses"(Fields et.al. 1987, p.5). The pre-technical program Afanufacturing 
Education Program (MEP) is offered mainly to the employees in the 
Electrical Products Division, and the basic skills program, 3TEP (Strategic 
Training and Education Program), is offered to maintenance employees so 
they can understand new automated machinery coming into the plant. 
The company hires instructors from the Minneapolis area school districts. 

The evaluation of Onan's Manufacturing Education Program (MEP) 
has "been limited to the assessment of course management and 
preparation solicited by instructors of students at the end of each 
quarter"(Fields, et.al. 1987, p.8). The students are given a form and are 
asked to assign the classes a grade from excellent to poor in the areas of 
administration, content, and applicability to Jobs. No program grade results 
were reported. Fields notes that as of yet, there is no attempt "to draw a 
direct relationship between basic skills and increased technical capacity of 
production efficiency" (p.8). However, Fields reports that the company has 
plans to interview program "regulars" in order to determine whether or 
not the content of the courses prepared them for more advanced classes. 
No mention is made of the evaluation process STEP undergoes. 
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Planter't Peanut Company 

The Planters Employee Training Program (PET) is one of the more 
established workplace literacy programs in the United States. In 1978, the 
Planters Company, the Suffolk City school system, and the Virginia 
Department of Education Joined forces to fight against the "low education 
level - over half, 53.2 percent, of the population in the city of Suffolk have 
not completed high school, according to the 1980 census"(Fields, et. al.. 
1987, p. 13). Program's objectives include helping employees "improve 
their reading, writing, and math skills in order that they may: 

* perform better in their present Jobs, 

* become eligible for promotion, 

* become better citizens of the community, and 

* experience the personal satisfaction that comes from 
leamlng"(p.l4). 

The only documentation supplied for program impact were anecdotes 
about learner Job-skill gains such as "A packer who was taught to read was 
able to determine where the boxes are going and to pack them 
appropriately"! p. 15). While this and other such reported gains were 
undoubtedly significant successes for the employees, to use them as sole 
evaluative measures is a weak indication of overall program effectiveness. 
Fields, et. al.(l987) report that further interviewing of PET participants 
indicated that some became more involved in the community and were 
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"able to print theif names and fill out forms, knowing for the first time 
what a stop sign means and being able to identify their name in print on a 
time card or check"(p.l5). No mention is made of the percentage of 
learners demonstrating such gains. 

Polaroid Corporation 

During the 1970's, Polaroid Corporation began a literacy program 
consisting of an ABE (Adult Basic Education) class, and ESL (English as a 
Second Language) class and a GED (General Education Development) class. 
The program is quite structured; there is a Fundamental SkiUi Program 
and a Technology Readiness Program which act as the preliminary bases 
for the more advanced Technician Training Program and the Polaroid 
Internal Technical Cooperative Program (PITCOP). Despite the highly 
structured organization of the literacy programs, "[mjuch of the program 
evaluation is informal; it consists of supervisors' acknowledgement of 
improved employee performance" (Fields et.al. 1987, p.20). No mention is 
made of the use of any structured methods to determine employee 
performance. The Technology Readiness Program is reported as 
developing "a formal evaluation process -- one that allows employees to 
reflect on learning" (p.20), but how employees will reflect and how that 
reflection will indicate program effectiveness is, again, not nientioned. 

RhJ. Reynolds Tobacco Company 

R. J. Reynolds Tobacco Company began offering ABE courses in 
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1983 in conjunctibn with the Adult and Continuing Education Division of 
Forsythe Technical College. The program, New Generation Training, 
focuses on improving basic skills, occupational skills, practical skills, and 
health-related needs. The classes are conducted in "direct instruction in 
an interactive mode between the instructor and one or two 
students"(Fields. et. al., 1987, p.23), and use the Laubach Way to Reading 
program. The Adult Basic Learning Examination (ABLE) was administered 
at both the start and end of the orogram. The scoring on the preprogram 
test determined in which class the learner would be placed and the 
post-program test scores were used to measure learner gains. No data 
were reported concerning these scores, their accuracy in determining 
learner needs, or the degree of learner improvements. There is no 
mention of any other evaluation techniques for either the learners or the 
prograi * itself. 

Rockwell International 

Rockwell became actively involved in the upgrading of basic 
education skills in 1978 when, in response to a perceived oil boom, it 
Joined a consortium of federal, state, community-based, and organized 
labor organizations to establish the Rocky Mountain Energy and 
Environmental Technology Center (RME and ETC). In 1983, the center's 
name changed to Technical Education Center (TEC) and it now works with 
the Community College of Denver and the Training, Administration, and 
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Assessment (TAA) office of Rockwell's Rocky Flats Plant. According to 
Fields, et. al.(1987), TAA is responsible for not only keeping training 
program records, but also the assessment of employees and the 
development and deliveiy of train-the-trainer classes. To date there has 
been no means of assessing TEC's effectiveness other than the "evidence" 
that union and management relations have reportedly improved because of 
the program. As a result of this strengthened relationship, TAA is planning 
"to develop a tracking system on employees in the program and then 
measuring results at six-month intervals for comparison with 
nonparticipants" (Fields etal., p.28). 
Canadian Programs 

Patterson (1989), for the Canadian Office of Planning Services, 
reviewed existing research and the current literacy training practices in 
Canada to draw conclusions and make recommendations for future literacy 
initiatives. Patterson reports that "(wjhile recent years have seen an 
increased amount of activity in Canada in the area of adult literacy, 
involvement in the area of workplace literacy is relatively new"(p.l 1). She 
briefly describes four of these programs: Frontier College's Learning in the 
Workplace Program, Lauhach Literacy of Canada's Industrialized Tutoring 
Program, Ontario's Basic Skills in the Workplace Program, and Douglas 
College's Basic Skills Upgrading Program. 

Patterson indicates that in 1985, a survey was sent to individuals 
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in the workforce aisking if they had ever participated in a Job-related 
training program. Only 8% of the people surveyed actually participated in 
a program. Patterson (1989) concluded that "there is little indication that 
this training benefited workers with lower skill levels"(p.l3). 

Summary of Evaluation Results 

Several trpnds are apparent from examination of current workplace 
literacy program descriptions and evaluations. 

1) Programs able to demonstrate effectiveness require significant 
resources in terms of learner time on task. Effective programs 
average from 50-100 hours of instruction per average 1 year of 
learner gain. Traditional business short courses are not 
particularly useful. Effective private programs report learner cost 
figures more than double those of public programs. 



2) Effective programs integrate basic skills training with workplace 
technical training. This usually occurs with an on-the-job 
training linkage and analysis of the basic skills needed on learner 
Jobs. Counseling is also integrated In the more effective programs. 



3) The majority of workplace literacy programs described in the 
available research literature tend to report no rigorous evaluation 
data. When programs are evaluated, assessments are often limited 
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to the completion of questiomiaires, surveys of program 
participants, and anecdotal reports of effectiveness. Occasionally a 
standardized reading test provides an indication of learner gain in 
general reading ability. 

4) Only a few evaluations provide follow-up data on the impact of 
programs on learner Job performance, retention, or earning 
power. 

Evaluation Plans 

The eight volume J'fPA evaluation plan (Washington State Dept., 1986) 
is the most thorough guideline for evaluation the effectiveness of vestibule 
literacy programs. It's very thoroughness, and the difficulty of keeping 
careful records and control group data in the face of federal counter 
incentives and limited resources may make such evaluations close to 
impossible, however. 

In addition, proponents of alternative assessment methods suggest that 
traditional assessment may miss essential aspects of program success. For 
example, Sterling (1989) points out that Headstart demonstrates its true 
effectiveness several years after children leave the program. In addition, 
traditional test-driven assessments miss gains in learner chosen goals, 
which can only be determined by careful and continued interview using 
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ethnographic methodologies. 

Not all program evaluations are published. Corporations with 
workplace literacy programs sometimes hire consultants to evaluate 
programs with the provision that results must be the sole property of the 
funder. Mikulecky and colleagues* have performed several such 
evaluations using a version of StufHebeam's (1974) Context, Input. Process. 
Product evaluation model modified for use with workplace literacy 
programs. 

In brief, the evaluation model employs the use of interviews, document 
analjrsis. observations, and test data to determine: 

1) the degree to which all involved with the program 
understand and share program goals; 

2) whether the resources in terms of personnel, materials, 
learning environmeni, and learner time are sufficient, 
given current knowledge, to achieve the goals; 

3) whether the learning processes and methods employed are 
sufficient to accomplish the goals; and 

4) what evidence exists that goals have actually been 

accomplished, 



• Mikulecky & Ehlinger with electronics occupations, Mlkulecl^ & Phillppi and Phillppl, 
MikuleclQr & Kloostemian with automotive occupations, Mlkuleclty & Helmsly with 
health service occupations. 



26 



How Effective Are Workplace Literacy Programs? 



Program Goals ; Interviews, analysis of memos and planning 
documents, and early program observations often reveal that significant 
differences about program goals exist among funders, supervisors. 
Instructors, materials designers and learners. Evaluation feedback during 
early program stages often Initiates necessary clarification among program 
planners and participants. In some cases goals are expanded, in some 
cases goals are refined, and in some cases new vendors are sought. 

Resources ; Early examination of resources often reveals that resources 
are insufficient to accomplish goals espoused by program planners. Tyrpical 
deficiencies are: 1) insufficient learner time to accomplish purported 
goals, 2) lack of learning materials or resources to develop materials which 
match the workplace literacy goals, and 3) hiring instructors with 
experience in general basic skills instruction but little knowledge or 
expertise about workplace literacy program development. 

Learning Methods : When instructors do not understand or share 
progrsim goals and resources are insufficient to meet these goals, 
observation of classes and interviews with learners are likely to reveal 
similar Inadequacies. Examples are: 1) Insufilcient learner practice time 
with literacy and too much class time allocated to discussion, 2) Teaching 



27 

30 



How Effective Are Workplace Literacy Programs? 

general reading Instruction with school books, off the shelf materials, or 
sometimes materials the instructor thinks will be of motivational interest 
to learners, and 3) Little feedback on learner accomplishments (often 
instructors are unable to comment upon what individual learners can and 
cannot do). Effective programs employ a mixture of workplace materials or 
modified workplace materials matched to Jobs, some general materials 
(especially for very low level readers), and sufQcient learner practice time 
to allow reasonable expectation of success. The learning time and materials 
often revolve around using reading, writing, computation, and some team 
work to solve problems similar to those encountered on the Job or at 
home. Some effective programs even manage to expand practice time 
through homework. 

Evidence of Goal Attainment ; Effective programs have gathered 
baseline data on the reading abilities and the reading practices of learners. 
This is accomplished through formal tests, informally constructed tests 
related to workplace expectations, and interviews with learners and 
sometimes supervisors. This information establishes a base for later 
comparisons. Later results can reveal that programs work most effectively 
for a particular ability level of learner. Some goals may be found to be 
achieved (i.e. Job reading gain and application of learning strategies to the 
Job), while other goals (i.e. general reading gain or educational attainment) 
may require more time. One program with which the author is currently 

28 



How Effective Are Workplace Literacy Programs? 

working is attempting to have entire work teams enter a combined 
technical training and workplace literacy training program. It is hoped 
that internal politics will allow evaluators to gather baseline data on defect 
rates, down time, and general productivity compared to teams not yet in 
training. 

The workplace literacy program evaluation model outlined above has 
the advantage of being both formative and summative. Potentially serious 
flaws in program design, which have been found in every program 
evaluated by the author thus far, can be addressed early while modification 
is still possible. In addition, the model involves program planners at the 
outset in gathering baseline data related to articulated program goals. This 
is often a political process. Sometimes difficulty in gathering baseline data 
unearths issues which must be dealt with before program delivery and 
success is likely. 

Evaluation cf workplace literacy programs involves a commitment on 
the parts of all involved to determine the degree to which programs are 
effective. This will require several changes in attitude and practice. 
Initially, sufilcient resources must be allocated for formative and 
summative evaluation. In the experience of this author, 2-3% of program 
development costs is a typical figure for effective evaluations. More 
complicated evaluations, of course, cost more. Attitude changes must also 
occur. Business leaders sometimes look at literacy programs as either a 
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charity expense or as something one sub-contracts to a vendor. Program 
quality will improve when funders view workplace literacy programs the 
same way the view any other cost of business which must be routinely 
evaluated and monitored. Vendors who are unable to perform are replaced 
in other areas of business. A degree in caring is not sufficient. Finally, 
vendors and instructors (often from local Adult Basic Education agencies) 
need to understand that workplace programs require a different way of 
Hoing business. Though the relationship between instructor and learner is 
still key, it is insufficient to guarantee program effectiveness. Business and 
union funders have every right to expect reasona.ble answers about how 
long gains will take, what resources will be required, and who will have 
access to evaluation information. Many program providers have never 
needed to consider such questions and indeed find such questions 
somehow distasteful. To become effective in this new area, program 
providers must instead adopt a trouble-shooting attitude toward their own 
programs. This means regularly asking: "What is working aixd what is not?" 
and "How do I really know?" 



30 

^3 



How Effective Are Workplace Literacy Programs? 



References 

Auspos, P. & Others. (1989). Implement Iny JOBSTART: A 

Demonstration for School Dropouts in the JTPA System . New 
York: Manpower Demonstration Research Corporation. ERIC 
Document Reproduction Service No. ED 311 253. 

Business Council for Effective Literacy, New York, N.Y. (1987) . 
Unions: Bread. Butter & Basir Skills. ERIC Document 
Reproduction Service No. ED 300 544. 

Fields, E. L., Hull, W. L., & Sechler, J. A. (1987). Adult 
Littarary; Indust-ry-Basf>d Training Programs . (Research and 
Development Series No. 265C) ERIC Document Reproduction 
Service No. ED 284 981. 

Haigler, K. (1990) The Job Skill's Education Program; An 

Exptqrimenf in Te chnology Transfer for WorkplarP Literacy A 
discussion paper prepared for the Work in America Institute, 
Harvard Club, New York, June 1990. 

Hargroves, J. (1S39) The basic skills crisis: One bank looks at 
its training investment. New England Economic Revie w. 
September/October, 1989, pp. 58-68. 

Hirschoff, P. (1988). Wldf^r Opportunitie s: Combining Literacy 
and Employment Trainin g for Women. Executive Summary of the 



31 



How Effective Are Workplace Literacy Programs? 



Female Single Parent Literacy Pro-io rt Case stnrtif^s . ERIC 
Document Reproduction Service No. ED 300 553. 

Mikulecky, L. & Strange, R. (1986) . Effective literacy training 
programs for adults in business and municipal employment. 
In J. Orasanu (ed.) Reading Comprehension: E^om Rf>search 
to Practice . Hillsdale, NJ; Lawrence Erlbaum Associates, 
Publishers, pp. 319-334. 

Patterson, M. (1989) Workplace Literac y! A Review of the 
Literature . Fredricton, New Brunswick: New Brunswick 
Department of Advanced Education. ERIC Document 
Reproduction Service No. ED 314 142. 

Philippi, J. (1989) U.S. Department of Labor Tfinhnnlogy 
Transfer Partnership Project: JESP App lication in the 
Private Sector. Lessons Learned Report. Washington, D.C., 
National Alliance of Business. 

Rosenfeld, S. A. & Others. (1987). Learning While Earning: 
Worksite Literacy Programs. Foresight: Model Programs for 
Economic Development . ERlC Document Reproduction Service No. 
ED 291 881. 

Sterling, R. (1989). Toward a model of alternative assessment. 
In Literacy and the Marketplace . New York: The Rockefeller 
Foundation. (A report on as meeting of practitioners, 
policymakers, researchers, and funders) , pp. 50-55. 

Sticht, T. G. (1982) . Basic Skills in Defense . Alexandria, 

32 
'^^^ 



ERIC 



How Effective Are Workplace Literacy Programs? 



VA: HumRRO. 

Stuf flebeam, D. (1974). Meta-Evalnal-ion ! Pap^r No. 3 . 

Occasional Paper Series. Kalamazoo: Evaluation Center, * 

College of Education, Western Michigan University. 
Washington Sate Department of Employment Security. (1986) . JTPA 

Evaluation at the State and Locr ' Levftl . Vnlntn^^ T; Ovprvif^w. 

ERIC Document Reproduction Service No. ED 278 837. 



